Real Time Validation of Online Situation Awareness Questionnaires in Simulated Approach Air Traffic Control

Measuring Situation Awareness to evaluate an operator’s ability to handle complex dynamic situations and the use of assistance systems have become a standard approach in Human Factors research. Ideally, the operator’s work should be supported by enabling and disabling assistance systems depending on how well they are currently able to keep track of the situation. On the one hand, if the situation’s complexity increases and therefore Situation Awareness is likely to be reduced, additional systems may help to overcome the operator’s limitations of cognitive resources such as working memory and attention by taking control over the specific task or task components. On the other hand, there has been evidence showing that an overuse of assistance systems, which reduces the task of the operator to mere monitoring, reduces his ability to intervene in a timely manner if needed. Adjusting the usage of assistance systems based on the operator’s Situation Awareness may help to overcome both limitations. However, existing measurement tools for Situation Awareness do not allow for an instant validation of the answers given by subjects. Instead, they require post-simulation analysis, e.g. replaying scenarios to compare the subject’s answers with the actual state of the situation. This way it is not possible to make decisions based on the operator’s Situation Awareness in real time, i.e., parallel with the situation at hand. Software capable of analyzing the current state of a given situation has been developed to overcome this limitation allowing real time assessment of Situation Awareness. This software was designed to measure Situation Awareness of approach air traffic controllers in the real time simulator NLR ATM Research Simulator. Situation Awareness was measured by online and offline probe questionnaires such as the Situation Present Assessment Method which were designed to be validated online as well. Before each item was presented, the simulation’s log files were analyzed to provide the software with the correct answer before the item was presented to the operator. This way, the answers given could be validated immediately allowing for an instant evaluation of the operator’s Situation Awareness. In a first study, a sample of 57 non-expert subjects was presented with online probe questionnaires in three different real time simulated approach air traffic control scenarios. It was found that the software was able to reliably analyze the answers given by the subject and to compare them with the actual situation. In the future, such systems could be used to make decisions about the need for further assistance while the situation is still happening. This way, operators would only get the necessary amount of assistance without reducing their work to passive monitoring. © 2015 The Authors. Published by Elsevier B.V. Peer-review under responsibility of AHFE Conference. 2 Jan Kraemer & Heinz-Martin Süß / Procedia Manufacturing 00 (2015) 000–000